Can Machines Read Coding Manuals Yet? – A Benchmark for Building Better Language Models for Code Understanding
نویسندگان
چکیده
Code understanding is an increasingly important application of Artificial Intelligence. A fundamental aspect code text about code, e.g., documentation and forum discussions. Pre-trained language models (e.g., BERT) are a popular approach for various NLP tasks, there now variety benchmarks, such as GLUE, to help improve the development natural understanding. However, little known how well work on textual artifacts we unaware any systematic set downstream tasks evaluation. In this paper, derive benchmarks (BLANCA - Benchmarks LANguage Coding Artifacts) that assess based predicting best answer question in post, finding related posts, or classes hierarchy from class documentation. We evaluate performance current state-of-the-art these show significant improvement each task fine tuning. also multi-task training over BLANCA build better
منابع مشابه
a new approach to credibility premium for zero-inflated poisson models for panel data
هدف اصلی از این تحقیق به دست آوردن و مقایسه حق بیمه باورمندی در مدل های شمارشی گزارش نشده برای داده های طولی می باشد. در این تحقیق حق بیمه های پبش گویی بر اساس توابع ضرر مربع خطا و نمایی محاسبه شده و با هم مقایسه می شود. تمایل به گرفتن پاداش و جایزه یکی از دلایل مهم برای گزارش ندادن تصادفات می باشد و افراد برای استفاده از تخفیف اغلب از گزارش تصادفات با هزینه پائین خودداری می کنند، در این تحقیق ...
15 صفحه اولa study on thermodynamic models for simulation of 1,3 butadiene purification columns
attempts have been made to study the thermodynamic behavior of 1,3 butadiene purification columns with the aim of retrofitting those columns to more energy efficient separation schemes. 1,3 butadiene is purified in two columns in series through being separated from methyl acetylene and 1,2 butadiene in the first and second column respectively. comparisons have been made among different therm...
proteomics a key tool for a better understanding of endometriosis:
endometriosis is a painful reproductive disease afflicting about up to 20% of women. it is one of the most frequent benign gynaecological diseases, however, little is known about the pathological of endometriosis. over the past decade, high-throughput proteomics technologies have evolved considerably and have become increasingly more commonly applied to the investigation of female reproductive ...
متن کاملModel Building for Natural Language Understanding
This paper introduces a discipline in the field of automated reasoning that has received too little attention so far within computational semantics: model generation or model building for first-order logic. Model builders offer a positive handle on the satisfiability problem (theorem provers are, in general, unable to detect the satisfiability of a problem) and are therefore often used in tande...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2022
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v36i4.20363